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1. Introduction 


1.1 Background 

The AH-64D Apache Longbow Block III upgrade program will provide aircrews with the 
capability to conduct level II (receive unmanned aerial system (UAS) sensor video), level III 
(control UAS sensor), and level IV (control of UAS sensor and air vehicle) interoperability with 
an UAS during missions. Aircrews will employ the UAS as a remote sensor, adding an 
additional sensor capability to the AH-64D reconnaissance and targeting systems. Testing has 
established that AH-64D aircrews can perfonn basic UAS operations from the crewstation 
during flight; however, there has been no comprehensive examination of the impact of UAS 
operations on aircrew workload. 

The U.S. Army Research Laboratory (ARL), Human Research and Engineering Directorate 
(HRED) conducted an assessment of mental workload that AH-64D aircrews experienced when 
controlling an UAS during simulated missions. The assessment was conducted to determine if 
the pilot (PI) and copilot-gunner (CPG) experienced tolerable workload while they controlled an 
UAS and performed standard flight and mission tasks (e.g., navigation, communication) during 
missions. The assessment also addressed the workload requirement listed in the AH-64D Block 
III (AB3) Capability Development Document (CDD). The CDD requirement is that “Longbow 
Block III crewstation management and cognitive decision aiding systems must provide a 
workload environment for the crew with a Bedford workload rating not to exceed 6.0 
(Threshold) and 5.0 (Objective).” 

1.2 System Description 

The AH-64D Apache is a twin-engine, tandem-seat, aerial weapons platfonn built by Boeing 
Integrated Defense Systems. Aircraft armament includes a belly-mounted slewable 30-mm chain 
gun, Hellfire missiles, and 2.75 in. aerial rockets. The aircraft integrated sensor suite includes a 
mast-mounted Longbow fire control radar (FCR) and a nose-mounted modernized target 
acquisition designation sight/pilot night vision sensor (MTADS/PNVS). There are two multi¬ 
purpose displays (MPDs) in each cockpit; the MTADS electronic display and control in the CPG 
crewstation; and the integrated helmet and display sight system. The aircraft has a flight control 
system with a fully articulated, four-bladed main rotor system. The flight control system consists 
of conventional cockpit controls: cyclic, collective, and pedals connected mechanically to 
hydromechanical actuators for the main and tail rotors; a limited authority automatic stabilization 
system; and an electrically actuated stabilator. 
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The Block III Apache is a modernized version of the AH-64D Block II Apache aircraft. The 
design incorporates technology advancements from other development programs and advances in 
processing technology since the fielding of the Block I and II AH-64D Apache. The planned 
upgrades should result in improved aircraft performance, reduced operating costs, and improved 
mission performance. One of the major upgrades is the provision for control of an UAS. 
Integration of the Tactical Common Data Link (TCDL) in the AH-64D will provide the aircrew 
with the capability of level II, level III, and level IV control of an UAS. 

1.3 Assessment Overview 

The assessment consisted of operational missions conducted by Apache aircrews in the 
AH-64D Risk and Cost Reduction Simulator (RACRS). The simulator was modified to 
represent the UAS Level 2-4 functionality to the maximum extent practicable based on the 
maturity of the Apache Block III design at the time of the assessment. 

Pilots received two days of training prior to the beginning of the assessment. The training 
consisted of classroom instruction and hands-on flight training in the RACRS. The pilots flew 
the same types of missions during training that they later flew during the record trials. The 
mission scenario was based on a battlefield environment simulating southwest Asia. Each 
successive mission increased in difficulty in order to impose progressively greater workload on 
the pilots. The aircrews performed specific Aircrew Training Manual (ATM) tasks during each 
mission (appendix A). Each ATM task had prescribed conditions and standards to which both 
crewmembers had to perform to help ensure mission accomplishment. 

During the formal evaluation, the aircrews perfonned Air Escort missions. The mission 
scenarios were developed by the Training and Doctrine Command (TRADOC) Capability 
Manager, Reconnaissance Attack (TCM RA) office in Fort Rucker, AL. The scenarios were 
developed in accordance with established aircraft tactics, techniques, and procedures (TTP). 

The pilots completed the Simulator Sickness Questionnaire (SSQ) before and after each flight. 
They also completed the Bedford Workload Rating Scale (BWRS), the Situation Awareness 
Rating Technique (SART), and the UAS-Crewstation Interface (UCI) questionnaire after each 
mission. During each mission, the CPG wore an eye tracker, which was used to assess pilot 
visual workload. In addition to the pilot data, subject matter experts (SMEs) provided an 
independent assessment of aircrew workload, situation awareness (SA), and mission success. 

The SMEs completed an aircrew workload, SA, and mission success survey after each mission. 

A mission debriefing and after-action review (AAR) were completed after each mission. 

During the simulation, aircrew actions within the cockpit were recorded for post-test analysis. 
Video recordings of each crewstation and all displays were kept as a permanent record. All 
button presses, switch activations, and MPD page displays were recorded during the simulation. 
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1.4 RACRS Cockpits 


The RACRS cockpits consisted of high fidelity aircraft flight controls and displays (figures 1 and 
2). The CPG used Target Acquisition and Designation System (TADS) Electronic Display and 
Control (TEDAC) grips to select and control the sensor’s field of view, azimuth, elevation, gain, 
and level. These controls were also selectable for adjustment of the UAS sensor. The TED AC 
and MPD displays were used to monitor the sensor view from the Apache and/or the UAS. 



Figure 1. Apache RACRS simulator (Camber Corp). 



Figure 2. Apache RACRS CPG cockpit (Camber Corp). 
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1.5 One Semi-Automated Forces (OneSAF) 


The OneSAF simulation provided the ability to generate threats and targets on the battlefield. 
Scenarios were developed that incorporated the detection, identification, and acquisition of 
simulated threats and targets. OneSAF produced outputs on the simulation network that enabled 
all connected simulations to receive and display the threats and targets. 


1.6 UAS Control Station 


A stand-alone workstation (figure 3) was developed to allow the UAS operator to independently 
fly an UAS during the scenarios. This was a desktop computer with visual system representing 
the same terrain location as the RACRS. A commercial joystick and keyboard provided user 
input for UAS control. The operator was linked with the Apache crew via audio 
communications using the existing lab intercom. The UAS positional information was output 
onto the simulation network allowing the other components (e.g., wingman) to be aware of its 
location. 



Figure 3. UAS control station (Camber Corp). 


1.7 AH-64 #2 Control Station 

A stand-alone workstation (figure 4), similar to the UAS control station, was developed to allow 
an operator to independently fly a second AH-64 (wingman) during the scenarios. This was a 
desktop computer with visual system representing the same terrain location as the Apache 
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simulation. A commercial joystick and keyboard provided user input for the second AH-64 
control. The operator was linked with the Apache crew via audio communications using the 
existing lab intercom. The AH-64 positional information was output onto the simulation 
network allowing the other components to be aware of its location. 



Figure 4. AH-64 #2 control station (Camber Corp). 


1.8 Terrain Location 

The simulator visual system was configured to fly the existing Bagram, Afghanistan, visual 
database (figure 5). This is a geo-specific large gaming area built from satellite acquired high- 
resolution imagery and detailed terrain relief. It also contained several terrain and cultural 
features to increase realism for the Apache pilots. 



Figure 5. Apache RACRS Afghanistan database screenshot (Camber Corp). 
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2. Method 


2.1 Data Collection 

Pilot workload, SA, crew coordination, UAS crewstation interface, switch actuations, simulator 
sickness, visual gaze and dwell times (head-eye tracker), audio-video, and TTP data were 
collected and analyzed. These areas were assessed to detennine if (1) pilot workload was 
tolerable when interacting with an UAS, (2) pilot workload in the AH-64D was higher, lower, or 
comparable to pilot workload when not interacting with an UAS, (3) pilots have adequate SA 
when interacting with the UAS, (4) the UAS control interface was easy to understand and 
navigate, and (5) pilots experienced simulator sickness symptoms. The data were used to 
recommend design improvements to the UAS menu system, training, and associated switchology 
and to refine TTP. 

The BWRS, SART, UCI, SSQ, and SME questionnaires were developed in accordance with 
published guidelines for proper fonnat and content (O’Brien and Charlton, 1996). A pre-test was 
conducted to refine the questionnaires and ensure that they could be easily understood and 
completed by the pilots and SMEs. 

The pilots completed the workload, UCI, and SA questionnaires after each mission. The pilots 
completed the SSQ before and after each mission. The SMEs completed workload, SA, crew 
coordination, and mission success questionnaires after each mission. Additional data were 
obtained from the pilots and the SMEs during post-mission discussions and AARs. 

Questionnaire results were clarified with information obtained during post-mission discussions 
and the AARs. 

2.2 Demographics 

A demographics questionnaire was used to collect basic information on each pilot’s experience 
and flight qualifications. The demographic data documented the range of pilot experience levels 
and qualifications. 

2.3 Assessment of Crew Workload 

A common definition of pilot workload is “the integrated mental and physical effort required to 
satisfy the perceived demands of a specified flight task” (Roscoe, 1985). It is important to assess 
pilot workload because mission accomplishment is related to the mental and physical ability of 
the crew to effectively perform their flight and mission tasks. If one or both pilots experience 
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excessively high workload while performing flight and mission tasks, the tasks may be 
performed ineffectively or abandoned. In order to assess whether the pilots are task-overloaded 
during the missions, the level of workload for each pilot must be evaluated. 

2.3.1 Bedford Workload Rating Scale 

The pilots completed the BWRS (appendix A) immediately after each mission to rate the level of 
workload that they experienced when performing flight and mission tasks. The tasks were 
selected because they were estimated to have the most impact on aircrew workload during the 
missions. 

The BWRS has been used extensively by the military, civil, and commercial aviation 
communities for pilot workload estimation (Roscoe and Ellis, 1990). It requires pilots to rate the 
level of workload associated with a task based on the amount of spare capacity they feel they 
have to perfonn additional tasks. Spare workload capacity is an important commodity for pilots 
because they are often required to perform several tasks concurrently. For example, pilots often 
perform navigation tasks, communicate via multiple radios, monitor aircraft systems, and assist 
the pilot on the controls with flight tasks (e.g., maintain airspace surveillance) within the same 
time interval. Mission performance is reduced if pilots are task saturated and have little or no 
spare capacity to perfonn other tasks. Integration of the UAS control with the AH-64D 
crewstation should help ensure that pilots can maintain adequate spare workload capacity while 
performing flight and mission tasks. 

2.3.2 Visual Workload 

An eye tracker was used during the evaluation to assess visual gaze and dwell times for the 
pilots. The data were collected to help determine how often the CPG was able to maintain visual 
focus outside the aircraft to assist with navigation (e.g., identification of terrain features), local 
security, terrain flight, etc., when having the added task of controlling an UAS. Visual gaze and 
dwell time data help identify whether pilots experience excessive visual workload or cognitive 
capture because they had problems interpreting information presented to them on the crewstation 
displays (for example). 

2.4 Assessment of Crew Situation Awareness 

SA can be defined as the pilot’s mental model of the current state of the flight and mission 
environment. A more formal definition is “the perception of the elements in the environment 
within a volume of time and space, the comprehension of their meaning, and the projection of 
their status in the near future” (Endsley, 1988). It is important to assess SA because of its 
potential to directly impact pilot and system performance. Good SA should increase the 
probability of good decision making and performance by aircrews when conducting flight and 
mission tasks in the AH-64D. 
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2.5 Situation Awareness Rating Technique 

The SART (appendix C) is a multi-dimensional rating scale for operators to report their 
perceived SA. The SART was developed as an evaluation tool for the design of aircrew systems 
(Taylor, 1989) and examines three components of SA: understanding, supply, and demand. 
Taylor proposed that SA is dependent on the pilot’s Understanding (U) (e.g., quality of 
information they receive), and the difference between the Demand (D) on the pilot’s resources 
(e.g., complexity of mission) and the pilot’s Supply (S) (e.g., ability to concentrate). When D 
exceeds S, there is a negative effect on U and an overall reduction of SA. The formula 
SA = U - (D - S) is used to derive the overall SART score. The SART is one of the most 
thoroughly tested rating scales for estimating SA (Endsley, 2000). 

2.6 UAS Crewstation Interface 

The UCI impacts crew workload and SA during a mission. A UCI that is designed to augment 
the cognitive and physical abilities of crews will minimize workload, enhance SA, and contribute 
to successful mission perfonnance. To assess the UCI, the pilots reported any problems that 
contributed to high workload and low SA at the end of each mission. They also completed a 
lengthy questionnaire at the end of their final mission. The questionnaire addressed usability 
characteristics of the UCI. 

2.7 Assessment of Simulator Sickness 

Simulator sickness has been defined as a condition where pilots suffer physiological discomfort 
in the simulator, but not while flying the actual aircraft (Kennedy, Lilienthal, Berbaum, Balzley, 
and McCauley, 1989). It is generally believed that simulator sickness is caused by a mismatch 
either between the visual and vestibular sources of information about self-motion, or between the 
sensory information (e.g., acceleration cues) presented by the simulator and the sensory 
information presented by the primary aircraft that the pilot operates. When the sensory 
information presented by the simulator does not match the aircraft, the pilot’s nervous system 
reacts adversely to the sensory mismatch and the pilot begins to experience discomfort. 
Characteristics of simulator sickness include nausea, dizziness, drowsiness, and several other 
symptoms (Kennedy et ah, 1989). It is important to assess simulator sickness because the 
discomfort felt by pilots can be distracting. Pilot distraction is one of the operational 
consequences of simulator sickness listed by Crowley (1987). If pilots are distracted by the 
discomfort they feel during missions, their perfonnance is likely to suffer. Additionally, the 
discomfort could influence the perceived levels of workload and SA that the pilots experienced 
during a mission. 
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2.8 Simulator Sickness Questionnaire 


The SSQ was administered to the pilots to estimate the severity of physiological discomfort that 
they experienced during missions and help assess whether they were being distracted by the 
discomfort. The SSQ (Kennedy, Lane, Berbaum, and Lilienthal, 1993) is a checklist of 16 
symptoms. The 16 symptoms are categorized into three subscales. The subscales are 
Oculomotor (e.g., eyestrain, difficulty focusing, blurred vision), Disorientation (e.g., dizziness, 
vertigo), and Nausea (e.g., nausea, increased salivation, burping). The three subscales are 
combined to produce a Total Severity score. The Total Severity score is an indicator of the 
overall discomfort that the pilots experienced during the mission. 


2.9 Subject Matter Experts 

Two SMEs observed each mission and rated crew workload, crew SA, and mission success. The 
SMEs provided an independent assessment of the workload and SA levels experienced by the 
crews. They also helped identify whether problems with crew workload or crew SA contributed 
to lack of mission success. 

The SMEs were pilots assigned to the TCM RA and the AH-64D program management office. 
They held the ra nk of CW4 and each had over 3000 h of flight experience. The pilots had 
substantial experience conducting attack missions and were familiar with the UAS mission. 

They observed each mission using a suite of monitors that showed all crewstation displays and 
the location of the aircraft, friendly forces, and enemy forces. They also listened to all audio 
communications between crewmembers and outside sources (e.g., wingman, ground 
commander) during the missions. A SME conducted an AAR with the pilots at the end of each 
mission. During the AAR, the SME reviewed the positive and negative aspects of the mission to 
provide instruction to the pilots and develop and refine TTP. 

2.10 Evaluation Design 

While the evaluation was operational in nature rather than experimental, multiple variables were 
controlled in order to maximize the validity of the conclusions regarding the areas of evaluation. 
Table 1 summarizes the variables that were controlled during the simulation. 

Tablet. Test variables. 


Factor 

Control 

Conditions 

Mission 

Constant 

Air assault 

Flight profile 

Tactically varied 

Nap-of-Earth (NOE), contour 

Light conditions 

Constant 

Day 

Scenario 

Constant 

Southwest Asia 

Crew 

Constant 

Maximize crew familiarity 

Seat position 

Varied 

Front, back 

Flight uniform 

Constant 

Air Warrior Gen 3 Combat-Basic 


9 




2.11 Overhead Cockpit Cameras 

An overhead camera was mounted in the front seat and a small camera was mounted on the 
glareshield in the back seat to record pilot actions. This aided in determining what the pilots 
were doing during different phases of the mission. The cameras had a time stamp added so that 
the elapsed time could be compared with other data that was collected. 

2.12 Switch Actuations 

When initiated, the data collection software collected switch actuations at a 10 Hz rate. The 
collected data were time stamped and written to a comma delimited log file. The log file was 
then post-processed to an Excel spreadsheet format. The data collection software provided an 
interface to allow the operator to initiate and cease collection during an executed run. 


2.13 Head and Eye Tracker System 


CPG visual gaze and dwell times were collected through a head and eye tracking system from 
Applied Science Laboratories (ASL). The ASL system was used because it was capable of 
integrating a laser head tracker to allow unrestricted head movement during data collection and it 
was compatible with the integrated helmet and display sight system (IHADSS) flight helmet. 

The ASL Eye-Head Package included a Model 501 eye tracker and an Ascension Laserbird head 
tracker. This technology allowed us to collect data that specified point of gaze with respect to 
stationary objects (e.g., MPDs) within the CPG crewstation. The ASL software allowed data 
collectors to continuously monitor the eye position of the pilots by crosshairs superimposed over 
live imagery (figure 6). The software also included a built-in analysis tool that allowed data to 
be viewed in tabular or graphical format. 



Figure 6. Eye tracker, pupil/camera monitors, and control panel interface. 
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2.14 Data Analysis 


Pilot responses to the BWRS, SART, SSQ, and UCI questionnaires were analyzed with means 
and percentages. Their responses to the BWRS, SART, and SSQ were further analyzed with the 
Wilcoxon Signed Ra nk Test (WSRT) to compare the ratings between the CPG and pilot and 
between ratings for the UAS missions vs. comparable non-UAS missions to determine if the 
differences were statistically significant (<.05). 

The eye tracker data were summarized by calculating the total percentage of fixations that 
occurred for the different areas of interest (AOIs). Six AOIs were created for the CPG: right 
MPD, left MPD, TED AC, keyboard, and kneeboard. Visual gaze and dwell times were also 
recorded for out-the-window. A final category, called “Other,” captured eye fixations not 
focused on a specific AOI. 

2.15 Evaluation Limitations 

The primary limitations included the small sample size of pilots (N=10) who participated in the 
assessment, the limited training they received (2 days), limited number of missions they 
conducted, and mix of AH-64D software versions in the RACRS. 

These limitations are not uncommon when replicating a complex aviation system in a simulator. 
However, the information and data listed in the Results and Summary sections of this report 
should be interpreted based on these limitations. Additional data should be collected during 
future simulations and tests to augment and expand the findings contained in this report. 

2.16 Participants 

Ten AH-64D pilots participated in the assessment. Six pilots were assigned to the 3-101 
Aviation Regiment, Fort Campbell, KY; two pilots were assigned to the 21 st Cavalry Brigade, 
Fort Hood, TX; and two pilots were assigned to the Forces Command (FORSCOM) HQ G-3 
Office, Fort McPherson, GA. Nine pilots held the rank of warrant officer (CW2 = 2 pilots, 

CW3 = 3 pilots, CW4 = 3 pilots, CW5 = 1 pilot) and one pilot held the rank of Captain. Four 
pilots were rated Flight Activity Category (FAC) 1 and six pilots were FAC 2, and eight pilots 
were Readiness Level (RL) 1 and two pilots were RL 3. They represented a broad range of 
experience with total flight hours from 650 to 3875 h. The relevant demographic characteristics 
of the pilots are shown in table 2. 

Table 2. Pilot demographics (N = 10). 


Summary of 
Demographic 
Characteristics 

Age 

(yrs) 

Flight Hours 
in AH-64D 

Total Flight Hours 
in Army Aircraft 

Mean 

36 

860 

1897 

Median 

37 

750 

1400 

Range 

25-43 

550-1620 

650-3875 
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3. Results 


3.1 Crew Workload—Average Bedford Workload Ratings for Flight and Mission Tasks 

The average Bedford workload rating (for all tasks) was 2.6 for the CPG and 2.9 for the PI 
(figure 7). These ratings indicate that the CPGs and Pis typically felt that workload was 
tolerable for the tasks and they had enough spare mental capacity for all desirable additional 
tasks. The difference in workload ratings between the CPG and PI was not statistically 
significant (WSRT, Z = -1.856, p = .093). This suggests that pilots perceived that workload 
levels were comparable for the CPG and PI during most missions. 


Pilot Bedford Workload Ratings 



□ CPG 
■ PI 


Figure 7. PI and CPG workload ratings. 

The estimated average workload ratings for comparable non-UAS missions (using MTADS 
and/or FCR only) was 3.0 for the CPG and 2.8 for the PI. These ratings indicate that the pilots 
typically believed that workload was tolerable for the tasks and they had enough spare mental 
capacity for all desirable additional tasks when performing comparable missions without 
employing a UAS. The differences in workload ratings between UAS missions and comparable 
non-UAS missions was statistically significant for the CPG (WSRT, Z = -2.840, p = .006), but 
not statistically significant for the PI (WSRT, Z = -.577, p = .774). The majority of pilots 
commented that the workload levels they experienced during UAS missions were comparable to 
workload levels they experienced during missions using only the MTADS and/or FCR. They 
reported that having to manage an additional sensor (UAS sensor) increased their overall task 
workload, but the SA provided by the UAS sensor typically decreased the workload required to 
detect and engage targets. They stated that (1) it was easier for them to detect and engage targets 
using the UAS sensor because of the steep visual aspect angle (“God’s Eye” view) that the UAS 
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sensor provided compared to the shallower visual aspect angle that the MTADS and/or FCR 
typically provides during missions; (2) the SA provided by the UAS sensor reduced the time 
required to detect and engage targets; and (3) while they had more tasks to complete because of 
the additional sensor, they had more time to complete target detection and engagement tasks 
because of the greater stand-off range that the UAS sensor provided vs. the stand-off range that 
the MTADS/FCR provides during missions. 

3.1.1 CPG Task Shedding 

The CPGs reported that there were very few tasks that they had to ask the Pis to perform (due to 
high CPG workload). ARL personnel observed only a few instances when the CPG asked the PI 
to perfonn a task because he was experiencing high workload. However, the CPGs occasionally 
shed tasks such as (responding to) radio calls, did not maintain positive control of the MTADS 
and let it drift into its limit stops, and asked for Level 4 control (but did not take Level 4 control) 
of the UAS because they were busy with other tasks. 

3.1.2 Impact of Workload on Aircrew Coordination 

The majority of Pis (80%) reported that the required level of crew coordination was higher 
during UAS missions than comparable non-UAS missions. They commented that having to 
manage an extra sensor increased their workload and required them to interact more with the 
CPG. The CPGs were split on whether having to manage an extra sensor increased the required 
level of crew coordination: 50% of CPGs reported that the required level of crew coordination 
was higher, 30% reported it was lower, and 20% reported it was comparable to a non-UAS 
mission. 

3.1.3 SME Ratings of Aircrew Coordination 

The SMEs provided ratings (figure 8) of how well the crews perfonned aircrew coordination 
tasks (e.g., positive communication) per Training Circular 1-210. The SMEs rated aircrew 
coordination during most missions as “Good” (50%) or “Average” (35%). 
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SME Ratings of Aircrew Coordination 



Figure 8. SME ratings for PI and CPG aircrew coordination. 


3.1.4 SME Workload Ratings 

SMEs provided an overall Bedford workload rating for each mission that they observed (figure 
9). The average SME Bedford workload rating (table 3) was 4.9 for the CPG and 4.3 for the PI 
for all missions. These ratings indicate that the SMEs believed that (1) workload was tolerable 
for the CPG and PI but that they had reduced spare workload capacity, (2) the CPG could not 
give the desired amount of attention to additional tasks, and (3) the PI had insufficient spare 
capacity for easy attention to additional tasks. The difference in SME workload ratings between 
the CPG and PI was not statistically significant (WSRT, Z = -1.806, p = .088). 


Pilot and SME Bedford Workload Ratings 



□ CPG 
■ PI 


Figure 9. Comparison of crew and SME Bedford workload ratings. 
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Table 3. SME Bedford workload ratings. 


Workload Ratings 

PI 

CPG 

Combined 

Average 

4.30 

4.90 

4.60 

Median 

4.00 

5.00 

4.50 

Standard Deviation (S.D.) 

1.49 

1.37 

1.43 


In previous simulations and operational tests that ARL has helped conduct, SMEs have typically 
rated pilot workload higher than the ratings provided by the pilots. SMEs have more information 
(e.g., location of all threat and friendly vehicles) available to them to assess pilot performance 
and workload than the infonnation that the pilots have available to them. SMEs are often more 
aware of pilot mistakes (that may be attributable to workload) than the pilots. The additional 
information that the SMEs have likely results in a more critical assessment of pilot performance 
and workload. This area requires further study to identify the reasons why SMEs typically rate 
pilot workload higher than the ratings provided by the pilots. 

3,1.5 Visual Workload 

Figure 10 shows the average percentage of time that the CPGs were visually focused on each 
AOI during the missions. The CPGs were visually focused on the right MPD (UAS sensor 
video) for 49% of the time during missions. They were visually focused on the left MPD 
(Tactical Situation Display (TSD)) for 19% of the time and the TEDAC (MTADS sensor) for 
14%. It is interesting to note that the CPGs typically spent only 6% of the time visually focused 
out-the-window during missions. The majority of CPGs (80%) reported on post-mission surveys 
that they were “inside” the cockpit more than during a non-UAS mission; however, they 
commented that the amount of time they were visually inside the cockpit was not much more 
than a comparable non-UAS mission. They stated that they did not believe that the amount of 
time they were visually inside the aircraft increased the probability of accidents. Two CPGs 
reported that not having the visual scene represented on the Helmet Display Unit (HDU) kept 
them visually inside the aircraft more than if they were able to use the HDU for tasks such as 
local security and navigation. 
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TEDAC 

14% 


Out-The-Window 

6% 


MPD 

19 % 


MPD 

49% 


Keyboard 

2% 


Kneeboard/ 
Printed Map 
6 % 


Other 4% 


Figure 10. CPG visual gaze and dwell times during UAS missions. 

One baseline mission (MTADS/FCR only, no UAS) was conducted to compare how much time 
the CPG was visually focused outside the aircraft during the Air Assault mission (figure 11). 

The CPG spent just 2% of the time visually focused out-the-window during the baseline mission. 
The baseline mission suggests that the amount of time that CPGs are visually focused outside the 
aircraft during comparable missions (e.g., day, visual flight rules) when employing the UAS and 
MTADS/FCR is similar to missions when employing only the MTADS/FCR. 
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Keyboard 
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Figure 11. CPG visual gaze and dwell times during baseline mission (non-UAS mission). 

The Pis were split when asked if they were “inside” the cockpit more than during a non-U AS 
mission: 50% of the Pis reported they were inside more than during a comparable non-UAS 
mission, 40% reported they were inside “about the same” as a comparable non-UAS mission, 
and 10% said they were inside less than a comparable non-UAS mission. A small camera was 
mounted on the glareshield in the rear cockpit so that ARL personnel could observe how much 
time the Pis spent visually focused inside vs. outside the aircraft. ARL personnel observed that 
the Pis were visually focused outside the aircraft approximately 75% of the time and inside 25% 
of the time (figure 12). The Pis confirmed the observations made by ARL personnel during 
post-mission discussions. 
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Out-The-Window 

75% 
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Observing Pilot Visual 
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In The Cockpit 


25°/( 


Figure 12. PI visual dwell times estimate. 

During the missions, the CPGs were not able to maintain visual focus outside the aircraft to 
assist with navigation (e.g., identification of terrain features), local security, terrain flight, etc. 

For 90%+ of the time, the CPGs were visually inside the aircraft perfonning (mostly) target 
detection and engagement tasks. It should be cautioned that the simulation was conducted with a 
small sample size of pilots, pilots did not get all of the peripheral visual cues that they would in 
an aircraft, pilots know that they cannot die if they crash the simulator (vs. aircraft), and only one 
baseline mission (MTADS only) was conducted. 

3.1.6 Comparison of Eye Tracker Data 

Table 4 shows a comparison of AH-64D, Armed Reconnaissance Helicopter (ARH), and UH- 
60M eye tracker data for Visual Flight Rules (VFR) flight during simulations. The ARH and 
UH-60M simulations were conducted to assess the human factors characteristics of the aircraft as 
part of system development and/or operational testing. While the aircraft, missions, training, and 
personnel experience levels were different for each simulation evaluation, it is interesting to note 
the differences in visual gaze and dwell times for each evaluation. 
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Table 4. Comparison of eye tracker results for AH-64D, ARH, and UH-60M simulations. 



AH-64D/UAS 

Workload 

Assessment 

UH-60M Early 
User 

Demonstration 2 

UH-60M Limited 
User Test 

UH-60M Limited 
Early User 
Evaluation 

ARH HFE-CAAS 
Evaluation 


Flying 

Pilot 3 

CPG 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

Non 

Flying 

Pilot 

Flying 

Pilot 

Non 

Flying 

Pilot 

Outside 

75% 

6% 

69% 

N/A 

86% 

28% 

61% 

26% 

61% 

7% 

Inside 

25% 

94% 

31% 

N/A 

14% 

72% 

39% 

74% 

39% 

93% 


“Estimate from watching PI gaze and dwell times with video camera during missions (non-eye tracker). 


3.2 Crew Situation Awareness—Situation Awareness Ratings 

The overall SART scores provided by the pilots were 24.0 for the CPG and 25.7 for the PI 
(figure 13). These scores indicate that the CPG and PI felt they had moderate levels of overall 
SA during the missions. The difference between SART scores for the CPG and PI was not 
statistically significant (WSRT, Z = -.612, p= 0.572). The estimated SART scores for 
comparable non-UAS missions (MTADS/FCR only) were 18.4 for the CPG and 23.2 for the PI. 
The difference between SART scores for the UAS missions and comparable non-UAS missions 
was not statistically significant for the CPG (WSRT, Z = -1.620, p= 0.133) or for the PI (WSRT, 
Z = -1.127, p= 0.289). The pilots stated that they had higher SA during UAS missions (vs. non- 
UAS missions) mostly because of the “God’s Eye” view that the UAS sensor video provided 
during missions. The UAS sensor video also gave the pilots good SA earlier in the mission (vs. 
non-UAS missions) because they often received the video prior (or just after) take-off of their 
aircraft. 


SART Scores For UAS Mission vs. Non-UAS 

Mission 



□ CPG 
■ PI 


Figure 13. Overall SART scores for PI and CPG. 
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The pilots reported that they had high levels of SA of most of the battlefield elements (appendix 
C) during the missions. The battlefield elements included location of enemy and friendly units, 
location of own ship, location of cultural features (e.g., bridges), and route information (e.g., 
waypoints). However, there were several instances when the aircraft flew near (or over) the 
target(s) and onboard missiles were fired outside of the aircraft/UAS constraints. This was likely 
caused by the lack of extensive training and experience with the UAS and the need for improved 
cueing symbology to help pilots understand where their aircraft and the UAS are located in 
reference to the targets. 

3.3 SME SA Ratings 

The SMEs provided an independent assessment of aircrew SA based on the scale shown in table 
5. The mean SMEs SA rating for aircrews was 2.40, indicating that the SMEs perceived that the 
aircrews typically had adequate levels of SA with minor or insignificant variation between 
perception and reality. The SMEs noted that there were several instances when the aircraft flew 
near (or over) the target(s) and onboard missiles were fired outside of the aircraft/UAS 
constraints. 


Table 5. SME SA rating. 



SME SA Ratings 

1 

Crew was consistently aware of all entities on the battlefield. 

2 

Crew was aware of the battlefield with minor or insignificant ^ 

variation between perception and reality. 

3 

Crew was aware of the battlefield. Variation between reality 
and perception did not significantly impact mission success. 

4 

SA needs improvement. Lack of SA had some negative effect 
on the success of the mission. 

5 

Lack of SA caused mission failure. 


Mean Rating 
2.40 

(SD = 0.99) 


3.4 SME Ratings of Mission Success and Mission Objectives 

At the end of each mission, SMEs rated whether the mission was a success or failure. The 
criteria that the SMEs used to rate mission success or failure was whether the aircrew completed 
most or all of their mission objectives and did not get shot down or crash. The SMEs rated all 10 
missions as “successful” (figure 14). They also rated whether the aircrew completed their 
mission objectives. The mission objectives were given to the pilots during the pre-mission brief. 
The SMEs believed that the aircrews completed their objectives during 90% of the missions. 

One SME rated two aircrews as not meeting all of their objectives during two missions. 


20 











SME Ratings of Mission Success & Mission Objectives 



Completed 


Figure 14. SME ratings of mission success and mission objectives. 


3.5 UAS Crewstation Interface 

The CPGs were generally favorable in their ratings of the UCI. They reported that they were 
able to quickly navigate through the UAS menu pages on the right MPD, quickly use the 
switches on the TED AC grips and buttons on the right MPD to control the UAS, felt that the 
overall design of the UCI did not significantly hinder them from controlling the UAS, and 
reported that the UCI did not contribute to high workload. The CPGs also reported that they 
“Never” (69%) or “Occasionally” (29%) forgot the steps required for navigating through the 
UAS display pages on the right MPD and experienced some problems switching between the 
operation of the MTADS/PNYS and UAS sensor due (mostly) to lack of experience. However, 
there were several improvements that the pilots recommended be made to the UCI to increase 
usability and decrease workload and the time required to complete tasks. These 
recommendations are listed in appendix D. 


3.6 Switch Actuations 

Switch actuations were recorded to help detennine whether there were CPG tasks (e.g., UAS 
menu navigation) that need to be streamlined because of excessive switch actuations. The CPGs 
made an average of 609 switch actuations per mission (appendix F). This equates to 
approximately 7-8 switch actuations per minute or one switch actuation per 8 s for each mission. 
The switch actuations were often clumped together within specific time intervals. The switches 
that were actuated most often by the CPG were the MTADS Field of View (FOV) select switch, 
right hand grip slave select switch, left hand grip tracker switch, left hand grip cursor up/down 
switch, right MPD bezel buttons, and right hand grip laser trigger. The pilots reported that they 
did not think that the number of switch actuations per mission was excessive. Many of the 
switch actuations were momentary actuations of the MTADS FOV select switch and right hand 
grip slave select switch. 
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Table 6 summarizes the number of MPD page changes per mission. The average number of 
right and left MPD page changes was 73 for the CPG and 69 for the PI per mission. The TSD 
and TCDL pages were the pages most often displayed by the CPG and PI during missions. 


Table 6. MPD page changes during missions. 



Number of Page Changes Per Mission 

Mission 

CPG 

(Right MPD) 

CPG 

(Left MPD) 

PI 

(Right MPD) 

PI 

(Left MPD) 

1 

24 

55 

28 

69 

2 

16 

36 

11 

34 

3 

50 

42 

8 

26 

4 

21 

54 

75 

35 

5 

52 

54 

13 

29 

6 

12 

44 

10 

39 

7 

12 

52 

41 

58 

8 

17 

45 

17 

18 

9 

11 

22 

25 

71 

10 

36 

76 

47 

32 






Avg 

25.1 

48 

27.5 

41.1 

Std. Dev 

15.53 

14.15 

21.36 

18.38 

Min 

11 

22 

8 

18 

Max 

52 

76 

75 

71 


3.7 Top Improvements Recommended by Pilots 

The pilots recommended that several improvements be made to the UCI and UAS to increase 
usability and decrease workload and the time required to complete tasks. Following are the most 
significant improvements that the pilots recommended be made to improve UAS employment 
(examples for each improvement are listed in appendix E): 

• Improvements to UAS Cueing Functions/Symbology 

• Improvements to Display Menu Pages 

• Need Linear Motion Control (LMC) for UAS 

• Incorporate UAS Operator into Pre-Mission Planning 

• Aviation Mission Planning System (AMPS) Integration of UAS Functionality 

3.8 Simulator Sickness 

Pilots reported that they typically experienced very mild to mild simulator sickness symptoms 
during the evaluation. The overall mean Total Severity (TS) score (post mission) for the pilots 
was 8.51 (table 7). The mean TS score for the CPGs was 9.72 and the mean TS score for the Pis 
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was 7.01. The difference between the TS scores for the CPG vs. the Pis was not statistically 
significant (WSRT, Z = -.210, p = .875). Based on the categorization of simulator sickness 
symptoms proposed by Kennedy et al (2002) (table 8), the pilots experienced “minimal” 
simulator sickness symptoms during the missions. 


Table 7. SSQ ratings. 


Condition 

Nausea 

Subscale 

Oculomotor 

Subscale 

Disorientation 

Subscale 

Total Severity 
Score (Mean) 

Standard 

Deviation 

Pre-Mission (CPG) 

.095 

4.54 

2.78 

3.36 

6.22 

Post-Mission (CPG) 

8.58 

9.09 

6.96 

9.72 

15.88 







Pre-Mission (PI) 

1.19 

0.94 

3.48 

1.87 

2.82 

Post-Mission (PI) 

9.54 

5.68 

1.74 

7.01 

11.28 







Pre-Mission CPG and PI 

1.06 

2.94 

3.09 

2.70 

4.93 

Post-Mission CPG and PI 

9.01 

7.58 

4.64 

8.51 

13.98 


Table 8. Categorization of simulator sickness symptoms. 


SSQ Total Score 

Categorization 

0 

No symptoms 

<5 

Negligible symptoms 

5-10 

Minimal symptoms 

10-15 

Significant symptoms 

15-20 

Symptoms are a concern 

>20 

A problem simulator 


Note: Categorization of symptoms based on central tendency (mean or median) 
using military aviation personnel in each simulator (Kennedy, 2002). 


3.9 Comparison of RACRS Simulator SSQ Scores to Other Helicopter Simulators 

To assess whether the SSQ ratings provided by the pilots during the assessment were similar or 
different to ratings obtained in other helicopter simulators, the mean SSQ scores for the RACRS 
simulator were compared to the mean SSQ scores for several other helicopter simulators (table 
9). The other helicopter simulators were the AH-64A (Army Research Institute, non-motion 
simulator), ARH-70, S-3H, CH-46E, CH-56D, CH-56F, Sikorsky RAH-66 Engineering 
Development Simulator (EDS), RAH-66 Comanche Portable Cockpit (CPC), and the simulator 
used during the UH-60M for the Early User Demo (EUD) and Limited Early User Evaluation 
(LEUE). In comparison, the RACRS induced fewer simulator sickness symptoms than most of 
the helicopter simulators listed in table 9. 
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Table 9. Comparison of RACRS simulator SSQ ratings with other helicopter simulators. 


Simulator 

Nausea 

Subscale 

Oculomotor 

Subscale 

Disorientation 

Subscale 

Total Severity Score 
(Mean) 

AH-64A 3 

- 

- 

- 

25.81 

ARH-70 

18.02 

21.48 

9.28 

20.15 

SH-3H 

14.70 

20.00 

12.40 

18.80 

RAH-66 EDS 

11.84 

14.98 

4.54 

13.25 

CH-53F 

7.50 

10.50 

7.40 

10.00 

RAH-66 CPC 

3.29 

12.94 

7.89 

9.80 

UH-60M (LEUE) 

6.36 

11.81 

3.09 

9.15 

RACRS 

9.01 

7.58 

4.64 

8.51 

UH-60M (EUD) 

13.88 

6.89 

0 

8.50 

CH-53D 

7.20 

7.20 

4.00 

7.50 

CH-46E 

5.40 

7.80 

4.50 

7.00 


a SSQ subscale data not available. 


Based on pilot SSQ ratings, observation by ARL HRED personnel during missions, feedback 
during post mission interviews, and comparison of SSQ ratings with ratings from other 
helicopter simulators, it is reasonable to assume that the simulator sickness symptoms the pilots 
experienced did not (1) cause them significant discomfort, (2) distract them during missions, or 
(3) contribute to an increase in perceived workload. 


4. Conclusion 


4,1 Crew Workload 

Pilots reported that they typically experienced tolerable workload when performing missions 
while controlling the UAS. They reported that the workload they experienced was comparable 
to workload they experience during “non-UAS” missions (MTADS and/or FCR only). They 
commented that having to manage an additional sensor (UAS sensor) increased their overall task 
workload, but the SA provided by the UAS sensor decreased the workload required to detect and 
engage targets and decreased overall target engagement timelines. The SMEs reported that the 
pilots typically experienced tolerable workload when controlling the UAS during missions, but 
had reduced spare workload capacity. The workload ratings provided by the pilots and SMEs 
were lower than the Objective and Threshold workload ratings requirements listed in the AB3 
CDD (table 10). 


Table 10. Pilot workload requirements and ratings. 


CDD Bedford Workload 

Pilot Bedford Workload 

SME Bedford Workload 

Rating Requirements 

Ratings 

Ratings 

Objective Req. - 5.0 

PI-2.9 

PI-4.3 

Threshold Req. - 6.0 

CPG-2.6 

CPG-4.9 
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4.2 Crew Situation Awareness 

Pilots typically experienced moderate levels of SA during missions. They reported that they had 
high levels of SA of most of the battlefield elements (e.g., threat location) during the missions. 
However, there were several instances when they flew near (or over) the targets and fired 
missiles outside of the aircraft/UAS constraints. This was likely caused by the lack of extensive 
training and experience with the UAS and the need for improved cueing symbology to help 
pilots understand where their aircraft and the UAS are located in reference to the targets. The 
pilots stated that they had higher SA during UAS missions (vs. non-UAS missions) mostly 
because of the “God’s Eye” view that the UAS sensor video provided during missions. The 
UAS sensor video also gave the pilots good SA earlier in the mission (vs. non-UAS missions) 
because they often received the video prior (or just after) take-off of their aircraft. The SMEs 
reported that the aircrews typically had adequate levels of SA. 

4.3 Crew Coordination 

The majority of Pis reported that the required level of crew coordination was higher during UAS 
missions than comparable non-UAS missions. They commented that having to manage an extra 
sensor increased their workload and required them to interact more with the CPG. The CPGs 
were split on whether having to manage an extra sensor increased the required level of crew 
coordination. The SMEs rated aircrew coordination during most missions as “Good” or 
“Average.” 

4.4 UAS-Crewstation Interface 

The CPGs were generally favorable in their ratings of the UCI. They reported that they were 
able to quickly navigate through the UAS menu pages on the right MPD, quickly use the 
switches on the TED AC grips and buttons on the right MPD to control the UAS, commented that 
the number of switch actuations per mission was not excessive, felt that the overall design of the 
UCI did not significantly hinder them from controlling the UAS, and reported that the UCI did 
not contribute to high workload. 

The pilots recommended that the following improvements should be made to enhance UAS 
employment: 

• Improvements to UAS Cueing Functions and Symbology 

• Improvements to Display Menu Pages 

• Need LMC for UAS 

• Incorporate UAS Operator into Pre-Mission Planning 

• AMPS Integration of UAS Functionality 
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4.5 Simulator Sickness 


Pilots reported that they typically experienced very mild to mild simulator sickness symptoms 
during the evaluation. The simulator sickness symptoms the pilots experienced did not (1) cause 
them significant discomfort, (2) distract them during missions, or (3) contribute to an increase in 
perceived workload. 


5. Recommendations 


The following recommendations are made to enhance the overall effectiveness and suitability of 
the UAS integration into the AH-64D: 

• Address and incorporate the recommended improvements (e.g., enhanced cueing) provided 
by the pilots. 

• Use the Crew Station Working Group to address and incorporate the recommended 
improvements. 

• Upgrade the RACRS simulator to make it representative of the AH-64D Block 3 design to 
enhance future simulations. 

• Maximize the amount of pilot training for future UAS evaluations. 

• Use the same data collection methodology (e.g., Bedford, SART) during future simulations 
and tests for Apache Longbow Block 3. Standardizing the data collection methodology 
will help identify changes that work (e.g., changes that reduce workload), identity areas 
that still need more work, amd help drive continuous incremental improvements. 
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Appendix A. Bedford Workload Rating Scale Scores and Pilot Comments 


Pilot Decisions 


Workload Description 


‘Rating’’ 


Workload insignificant 

1 

Workload low 

2 

Enough spare capacity for all 
desirable additional tasks 

3 


Was workload 
satisfactory 
without reduction in 
spare (workload) capacity? 


NO 


Insufficient spare capacity for easy 
attention to additional tasks 

4 

Reduced spare capacity. Additional 
tasks cannot be given the desired 
amount of attention 

5 

Little spare capacity: level of effort 
allows little attention to additional 
tasks 

6 


Was workload tolerable 
for the task? 


NO 


Was it possible to 
complete the task? 


NO 


Very little spare capacity, but 
maintenance of effort in the primary 
tasks not in question 

7 

Very high workload with almost no 
spare capacity. Difficulty in 
maintaining level of effort 

8 

Extremely high workload. No spare 
capacity. Serious doubts as to ability 
to maintain level of effort 

9 


Task abandoned. Pilot unable to 
apply sufficient effort 


Figure A-1. The BWRS. 
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Table A-l. Mean Workload Ratings for UAS missions and comparable non-UAS missions. 


Flight and Mission Tasks 

Mean Workload Ratings for 
UAS Missions 

Mean Workload Ratings for 
Comparable Non-UAS 
Missions 


CPG 

Pilot 

CPG 

Pilot 

Observing Named Areas of Interest 

2 

2 

3 

3 

Target Detection 

2 

3 

4 

3 

Target Acquisition 

3 

3 

4 

3 

Target Engagement 

3 

3 

4 

3 

Movement To Contact 

3 

3 

3 

3 

Actions On Contact 

4 

4 

4 

3 

Battle Damage Assessment 

And Reporting 

2 

2 

3 

3 

Mission Change 

3 

2 

3 

3 

Battle Handover 

2 

3 

2 

3 

Tactical Navigation (Contour/NOE) 

2 

3 

2 

2 

Communications (Tactical Ops Center, Wingman) 

2 

3 

2 

2 

NOE/Contour/Low Level Flight 

2 

3 

2 

2 

Maintain Airspace Surveillance 

3 

3 

3 

3 

VMC Flight Maneuvers 

2 

2 

2 

2 

Electronically Aided Navigation 

2 

2 

2 

2 

Terrain Flight Navigation 

2 

3 

2 

2 

Evasive Maneuvers 

3 

5 

4 

4 

MTADS/PNVS Operations 

3 

2 

3 

2 

Route Recon 

2 

3 

3 

3 

Area Recon 

2 

3 

3 

3 

Level 2 UAS Control 

2 

3 

- 

- 

Level 3 UAS Control 

3 

3 

- 

- 

Level 4 UAS Control 

3 

2 

- 

- 

Data Entry Procedures 

2 

3 

2 

3 

Engage with Hellfire 

3 

4 

3 

4 

Engage with 30mm AWS 

4 

2 

4 

2 

Multi-ship Operations 

3 

3 

3 

3 

Transmit Tactical Reports 

3 

3 

2 

3 

Identify Major U.S./Allied and Threat Equipment 

3 

3 

4 

3 

Information Management In The Front Seat (CPG) 

3 

- 

3 

- 

Information Management In The Back Seat (PI) 

- 

4 

- 

3 

Air Escort Mission (overall) 

3 

3 

3 

3 

Security Mission (overall) 

3 

3 

4 

3 

Overall Workload Ratings 

Avg: 2.6 
Median: 3 

Std. Dev.: 0.6 

Avg: 2.9 
Median: 3 
Std. Dev.: 0.6 

Avg: 3 
Median: 3 

Std. Dev.: 0.7 

Avg: 2.8 
Median: 3 
Std. Dev.: 0.5 
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Pilot Workload Comments 


If you gave a workload rating of‘5’ or higher for any task in the UAS mission column, explain 
why the workload was high for the task. 

Explanations as to why the pilots assigned a rating of ‘5’ or higher for any task in the UAS 
column. 

Evasive Maneuvers: 

• Task intensive, there is no way to add additional tasks and still be capable of performing or 
reacting to evasive maneuvers. 

• EM too task intensive to also be using UAS. Once completed or out of danger area and 
continuing to engage it becomes much easier. 

• EM caused CPG to totally concentrate on UAV while physiological functions and visual 
cues in the cockpit caused visual rivalries. 

Actions on Contact: 

• Task intensive, there is no way to add additional tasks and still be capable of performing or 
reacting to actions on contact. 

• All concentration goes to maintaining LOS centered on target and communicating. 

• AoC too task intensive to also be using UAS. Once completed or out of danger area and 
continuing to engage it becomes much easier. 

• Weapon engagements are always intensive. 

• Difficulty maintaining SA with wingman/UAV in relation to self when engaging TGTs. 
Target Detection/Acquisition/Engagement: 

• All concentration goes to maintaining LOS centered on target and communicating. 

• Concentrated effort of controlling the TFC (Thumb Force Control) on OBJ/TGT to 
maintain LOS during Helffire engagement. 

• Workload is lower for detection/identification/acquisition and engagement using the UAS. 
Crew can create a 3-D picture of the battle space vs. the 2-D (TADS) picture. 

• CPG remains focused on TGT with UAV unable to perfonn other tasks until engagement is 
complete. 

• TGT detection and Acquisition are easier with UAS; TGT engagement has a higher 
workload due to helffire designation basket. 
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Hellfire/Rockets/3 Omm: 


• All concentration goes to maintaining LOS centered on target and communicating. 

• Increased workload due to remote hcllfire basket. 

Movement to Contact: 

• Weapon engagements are always intensive. 

MTADS/PNVS Operations: 

• Note: Full MTADS (Multi-target tracking) not available. MTADS layout when controlling 
UAS. TFC directly located under the UAS FOV selector. Not ergonomically designed 
causing higher workload during TGT engagement (Target Tracking) using the TFC. 

Multiship Operations: 

• Unable to concentrate on wingman’s location. Relied on PI for SA. 

Information Management: 

• Still feels like the CPG has a large workload that can be managed with an opposite crew 
member. 

Mission Change: 

• MSN changes are easier with UAS than with TADS. 

• Increased workload due to decision-making process on how to maximize employment of 
UAS. 

Other Comments: 

• Processing the additional information the UAS provided, there was a heightened sense of 
urgency because this mission involved friendly forces on the objective. 

• Difficult processing information from extra sensor. 

• With legacy target ID it’s (mission)?? darn near impossible. 

List any flight and/or mission tasks that you had to ask your crewmember to accomplish 
because your workload was too high. 

PI Comments: 

Other Tasks: 

• Workload shared by crew based on task(s). 
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CPG Comments: 


Flight Maneuvers/Airspace Surveillance: 

• There were none he “had” to do, but the PC aided in many tasks. 

• Ah fit maneuvers and 90% airspace surveillance accomplished by PC. 

Communication Tasks: 

• Some communication (radio calls). 

• I could have done it, but knew PC had less to do and could offer assistance. 

• Change fireq, change direct, de-was missile. 

• Division of communication duties. 

Tactics: 

• Tactics (TMPR): we as a crew discussed this plan prior to initiating the maneuvers. I relied 
on the PI to keep my ACFT safe from wingman in relation to UAV. 

Target handover/transfer: 

• Briefed with PI, that he would transfer/handover TGTs I stored to wingman or ARTY to 
relieve the workload on CPG. This did assist, but I felt I had enough iterations in this 
profile/mission that I could have assisted with this task. 

Hellhre/Rockets/3 0mm: 

• 2-Hellhre missile shots. Busy assessing BDA. 

Were you ‘inside 9 the cockpit more, less, or about the same compared to a non-UAS mission? 

PI Comments: 

• Video from UAV and ensuring LOS of laser is in constraints to fire a hellfire requires a 
little bit more inside time, but not enough to adversely affect the ability of the pilot to 
remain in control of the aircraft. 

• I disciplined myself outside the acft due to the CPG increased time inside. 

• Helping front seat line up targets, acquire targets draws you inside similar to TADS shots 
when back seat is watching tads video on MPD. 

• Only inside more when CPG was loosing SA. After SA was reestablished I was back 
outside. 

• Processing UAS information. 
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• Heads out more today based on MSN. Some head-in time was trying to figure out what was 
wrong with HMD symbology-RACRS (sim) issues. 

• More inside to correlate UAS data with other sensors. 

CPG Comments: 

• No HDU. If I had UAV video in HDU would split more attention outside. 

• A little more along the route recon with the mountains. I would have slaved TADS to my 
helmet and reconned by eye. The UAS allowed me to recon using it instead. Better recon, 
but inside more. 

• Obviously inside more because of the third sight. However , after my 4 th fit. Using the 
system, this is becoming much easier and more familiar. With a little time, this system will 
be integrated into an Apache mission without any ill effects. 

• No HDU feed, so all CPG msn tasks involved sights viewed on MPD. 

• Taking advantage of greater stand-off distance to accomplish task of DIL (Detecting, 
Identifying, and Locating)/ Acquiring targets using sights and sensors. 

• Info management: CPG task saturated. 

• Level 3 control coupled with the security mission required CPG to have eyes on for the 
duration of the mission. 

• Tracking Targets. 

Was the required level of crew coordination higher, lower or about the same as a comparable 

non-UAS mission? 

PI Comments: 

• Inside cockpit a little more and coordinating our runs on target require more talking 
between crew members in order to ensure first time engagement success. 

• Increased workload for CPG drives pilot to ensure CPG is not task overloaded. This is done 
by offering assistance. 

• Drawn inside, so more cross talk between crews, but SA was highly increased on objective 
area so it seemed well worth it. 

• Back up CPG when his workload increases. 

• Extra sight, more information, weapons engagements are more complex with UAS as the 
sight. 
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• Additional sight adds to workload and crew coordination - comparable to FCR. However, 
SA is much greater with UAS than without and somewhat better than FCR. 

• As with previous missions, weapons engagements were more difficult due to FS controlling 
other sensor. 

CPG Comments: 

• Higher because there were more tasks to complete. Yet had more time to complete tasks 
due to increased ranges and SA. 

• Better SA in both seats equals less required communication, other than simple 
acknowledgement of same SA. 

• More steps and tasks to complete, yet more time and safer distance to do them. 

• Better SA for both seats = less workload. 

• Lower because of a good crew briefing to include UAS ground operator. 

• Keeping SA. 

• Keeping weapons MGMT/SA between wingman and UAV. 

• We are still having to put the AH64 wpns in constraints with UAS. Coordination is a must. 

• Change of plan/mission - Aircraft positioning for proper target engagements. 

In the mission you just flew, list any flight and/or mission tasks that you had to ask your 
crewmember to accomplish because your workload was too high: 

• There were none he “had” to do, but the PI aided in many tasks. 

• All fit maneuvers and 90% airspace surveillance accomplished by PI. 

• I could have done it, but knew PI had less to do and could offer assistance. 

• Change fireq, change direct, de-was missile. 

• Some communication (Radio Calls) 

• Tactics (TMPR): we as a crew discussed this plan prior to initiating the maneuvers. I relied 
on the PI to keep my ACFT safe from wingman in relation to UAV. 

• None 

• Workload shared by crew based on task(s). 

• Briefed with PI, that he would transfer/handover TGTs I stored to wingman or ARTY to 
relieve the workload on CPG. This did assist, but I felt I had enough iterations in this 
profile/mission that I could have assisted with this task. 
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Division of commo duties. 


• 2-Hellfire missile shots. Busy assessing BDA. 


In the mission you just completed, were you visually ‘inside ’ the cockpit more, less, or about 
the same compared to a comparable non-UAS mission (figure A-2))? 


Visually Inside The Cockpit vs. Non-UAS Missions 




Figure A-2. Visually inside the cockpit vs. non-UAS missions. 

• Video from UAV and ensuring LOS of laser is in constraints to fire a hellfire requires a 
little bit more inside time, but not enough to adversely affect the ability of the pilot to 
remain in control of the aircraft. 

• As the backseater, I disciplined myself outside the acft due to the CPG increased time 
inside. 

• A little more along the route recon with the mountains. I would have slaved TADS to my 
helmet and reconned by eye. The UAS allowed me to recon using it instead. Better recon, 
but inside more. 

• Obviously inside more because of the third sight. However , after my 4 th fit. Using the 
system, this is becoming much easier and more familiar. With a little time, this system will 
be integrated into an Apache mission without any ill effects. 

• Helping front seat line up targets, acquire targets draws you inside similar to TADS shots 
when back seat is watching tads video on MPD. 

• No HDU feed, so all CPG msn tasks involved sights viewed on MPD. 

• Only inside more when CPG was loosing SA. After SA was reestablished I was back 
outside. 
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• Taking advantage of greater stand-off distance to accomplish task of DIL (Detecting, 
Identifying, and Locating)/ Acquiring targets using sights and sensors. 

• Processing UAS information. 

• Info management: CPG task saturated. 

• Heads out more today based on MSN. Some head-in time was trying to figure out what was 
wrong with HMD symbology-RACRS (sim) issues. 

• Level 3 control coupled with the security mission required CPG to have eyes on for the 
duration of the mission. 

• No HDU. If I had UAV video in HDU would split more attention outside. 

• Tracking Targets. 

• More inside to correlate UAS data with other sensors. 


Was the required level of crew coordination higher, lower, or about the same as a comparable 
non-UAS mission? 

Much Higher level of crew coordination required - Front seat - 10% Back seat - 20% 
Somewhat Higher level of crew coordination required - Front seat - 40% Back seat - 60% 
About the Same level of crew coordination required - Front seat - 20% Back seat - 20% 
Somewhat Lower level of crew coordination required - Front seat - 30% Back seat - 0% 
Much Lower level of crew coordination required - Front seat - 0% Back seat - 0% 


Pilot comments: 

• Inside cockpit a little more and coordinating our runs on target require more talking 
between crew members in order to ensure first time engagement success. 

• Higher because there were more tasks to complete. Yet had more time to complete tasks 
due to increased ranges and SA. 

• Increased workload for CPG drives pilot to ensure CPG is not task overloaded. This is done 
by offering assistance. 

• Better SA in both seats equals less required communication, other than simple 
acknowledgement of same SA. 

• Almost same reason as 8. Drawn inside, so more cross talk between crews, but SA was 
highly increased on objective area so it seemed well worth it. 
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• More steps and tasks to complete, yet more time and safer distance to do them. 

• Back up CPG when his workload increases. 

• Better SA for both seats = less workload. 

• Lower because of a good crew briefing to include UAS ground operator. 

• Extra sight, more information, weapons engagements are more complex with UAS as the 
sight. 

• Keeping SA. 

• Keeping weapons MGMT/SA between wingman and UAV. 

• Additional sight adds to workload and crew coordination - comparable to FCR. However, 
SA is much greater with UAS than without and somewhat better than FCR. 

• We are still having to put the AH-64 wpns in constraints with UAS. Coordination is a 
must. 

• Change of plan/mission - Aircraft positioning for proper target engagements. 

• As with previous missions, weapons engagements were more difficult due to FS controlling 
other sensor. 
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Appendix B. Subject Matter Expert Workload Comments 


If you assigned a workload rating of ‘5’ or higher for either crewmember, explain why: 

• Front seater shed tasks to operate UAS. 

• Backseater - had good situation awareness. However, backseat was looking inside at the 
UAS video display during level 3&4 control to help the front seater find the targets. 

• The front seat is 90% or better inside during the mission. 

• Crew continuously flies within 1KM of the target area. 

• Requested level 4, never took control. 

• During pick up system initiation. 

• CPG - did not call BN Ops back during run-up for operation update. 

• CPG - launched two missiles at tgt3 -> Planned rapid but sim poor video feedback. 

• CPG - Asked for LVL 4, which was approved, but CPG did not take LVL 4 - remained 
LVL 3. 

• Did crew decide to remain LVL 3 - No - CPG perceived he was in LVL 4 guide mode. 

• Understanding what the UAV was doing under Level 4 control. Guide made us PROG 
mode - caught it after first missile. Reprogrammed a plan and then CPG selected PROG. 

• Crew did not know Level 4 was in guide mode, not plan mode. 

• WL-low along route to and from objective (good). Changed LVL 3 to LVL 2 to reduce 
workload shifted route recon to UAS operator during MSN change - allowed CPG to 
setup next MSN. Remained LVL 3 control for most of the mission. CPG initially checked 
WPN page prior to engagement - did not check WPN page as mission progressed CPG 
decided this check was not required prior to shot Omitted task during last half of MSN. 

• Front seat shed engagement tasks of storing TGTS and had difficulty re-acquiring targets. 

• WAS/De-WAS weapons 

• LOAL LO-DIR confusion (little) 

• FXD ACQ - and driving the missile constraints bey I AT FOV TGT drives the sight when 
beyond the FOV 
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• Storing TGT not practiced - if done, this would have reduced workload of CPG having to 
re-acquire. 

• One sim- problem increased WL when CPG tried to drop track the UAS FOV went to 
zoom causing the CPG to lose the TGT. 

• Hcllfirc remote excusum and designator zones exceeded 

• Crew tactic (launch UAV early) gave more time to work the targets and contributed to less 
stress. 

• Crew members had difficulty determining their position with the position of the UAS with 
the targets. 

• PLT loitered over the targets. Workload seemed low, but crew had an SA breakdown. 

• Aircrew lost SA and flew over targets and got shot at. Crew did not maintain standoff. 

• When aircrew had level 3 and 4 crew let sensor drift with no control. Crew did not use 
UAS ground controller to search for and designate targets. Crew shed the task of 
controlling UAS when they had level 3 or 4. 

• PLT performed HF MSL engagements and flying ownship entered MBT WEZ several 
times while maneuvering. 

SME Aircrew Coordination Comments: 

• Crew coordination continues to improve from previous missions. 

• The pilot (back seat) in this crew worked mission priorities and kept the CPG directed. 

• Crew briefed engagement plan - first TGT LVL2 control - TGT destroyed. 

• What were you using to line up MSL constraints? TGT as an ACQ source. 

• Real good job working the cockpits. 

• Very good division of Com duties. 

• Excellent division of duties between MUM team. 

• Very good SA in both crewstations. 

• Excellent crew coordination and division of duties. Assigned tasks to UAS operator when 
appropriate. 

• Aircrews fired multiple missiles outside the missile constraints - payload tracking - 
aircrew had difficulty tracking targets with UAS payload. 

• PL enabled CPG to have a lower workload by performing tasks. 
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Describe any problems that aircrews had with situation awareness: 

• TADS Operations were shed to operate UAS. 

• Did not know if they had level 4 control or not. 

• CPG fired second MSL with TADS FXD FWD - did you use UAS distance and HOG for 
MSL constraints used TGT as an ACQ source. 

• Crew closed to under 1KM during gun engagement. Crew discussion to pursue target. 

• Circled closer to EA-1-3KM from UAV LOS. 

• Crew awareness of UAV location 64 - TGT - UAS location. 

• Crew lost SA on UAV location during mission during missile engagements. Missiles 6, 7, 
and 8 were all outside the laser offset performance. 

• Crew fired 3 missiles outside the 60deg gun target line. 

• Front seat had problems knowing whether or not he had lased a weapon during missile 
engagements. 

• Using the UAS acq should have been used to maintain TGT position - lase and store. 

• Maintained excellent SA on the battle and other elements. 

• Crew was prompted to hurry in when AA force holding. 

• Crew prompted during AWS running foe by wingman to hurry up. 

• Could not properly detennine there position and the position of the UAS prior to missile 
engagements. 

• HF missile engagement with AATF further downrange along GTL. 

• Crew members had difficulty tracking with UAS payload and the crew had difficulty 
determining the location in regards to the UAS. 

• Loitered over the targets. 

• Crew over flew enemy targets. Did not maintain standoff. 

• Aircrew flew within 1km of targets. 

Did the aircrew complete their mission objectives? 

If no, why weren’t the mission objectives completed? 

• Enemy forces still alive after AA inserted delayed AAGF movement to objective. 

• Crew was engaged by enemy at known location. 
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Intentionally Left Blank. 
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Appendix C. Situation Awareness Ratings and Comments 


C-l. CPG 


‘Demand’ During Missions 


13.40 

T - 


Low 1-3-5-7-9-11-13-15-17-19-21 High 


‘Supply’ During Missions 

20.50 

Low 1-4-7-10-13-16-19-21-24-28 High 


‘Understanding’ During Missions 


16.90 

~T 


Low 1-3-5-7-9-11-13-15-17-19-21 High 


SART SCORE: 24.0 
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C-2. PI 


‘Demand’ During Missions 


13.70 

T~ 


Low 1-3-5-7-9-11-13-15-17 


19 


■21 High 


‘Supply’ During Missions 


Low 1-4-7-10 


21.80 

- ? - 

13-16-19-21-24 


■28 High 


‘Understanding’ During Missions 

17.60 


Low 1-3-5-7-9-11-13-15-17-19-21 High 


SART SCORE: 25.7 
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Table C-l. SA of battlefield elements during missions. 


Battlefield 

Elements 

Very High 
Level of 
Situation 
Awareness 

Fairly High 
Level of 
Situation 
Awareness 

Intermediate 
Level of 
Situation 

Awareness 

Fairly Low 
Level of 
Situation 
Awareness 

Very Low 
Level of 
Situation 
Awareness 


PI 

CPG 

PI 

CPG 

PI 

CPG 

PI 

CPG 

PI 

CPG 

Location of 
Enemy Units 

70% 

80 % 

30% 

10% 

0% 

10% 

0% 

0% 

0% 

0% 

Location of 
Friendly Units 

30% 

70 % 

60% 

10% 

10% 

20% 

0% 

0% 

0% 

0% 

Location of My 
Aircraft During 
Missions 

90% 

80% 

0% 

20% 

10% 

0% 

0% 

0% 

0% 

0% 

Location of 
Other Aircraft 

In My Flight 

30% 

10% 

40% 

40% 

30% 

30% 

0% 

20% 

0% 

0% 

Location of 
Cultural 
Features (e.g., 
bridges) 

50% 

30% 

40% 

40% 

10% 

10% 

0% 

20% 

0% 

0% 

Route 

Information 
(ACPs, BPs, 
EAs, RPs, etc.) 

90% 

80% 

10% 

20% 

0% 

0% 

0% 

0% 

0% 

0% 

Status of My 
Aircraft Systems 
(e.g., fuel 
consumption) 

70% 

0 % 

20% 

20% 

10% 

60% 

0% 

10% 

0% 

10% 


Note: BP = Battle position, EA = Engagement area, ACP = Air control point, and RP = Release point. 
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C-3 Aircrew Situational Awareness Comments 

Describe any instances when you felt you had low situational awareness during the mission: 

• Little airspace surveillance, busy mission not a lot of attention to acft systems status other 
than sights weapons and radios. 

• Only because of probable computer sim limitations. Non-combatants and friendly forces 
not in a vehicle or building made it hard to detennine where exactly they were at. 

• Not sure where our flight escorts were and I had very little aircraft info because I did not 
have my HDU. 

• None other than possible convoy fireq and location at end. 

• Wasn’t monitoring acft sys or wingman much during intensive engagements. 

• Difficulty finding last vehicle due to simulator issue. *** COMMENT*** Not familiar 
with Lot 4 load. 

• Searching for last target vehicle. 

• During relief-on-station due to lack of exercising the task. 

• No view on non-combatants. 

• CPG was too involved with mission, Required PI to update me on wingman location. 

• Cultural features: I did not bother to make note of them. I was too focused on mission to 
care! 

• *No digital capability in BFT in simulator. 

• I did not hear the UH A/C depart the holding area for the LZ. Not untypical for the Lift 
folks MGT to make that call regardless of a UAS. They will come in based on their fuel or 
the ground CMDR in the A/C ordering them to do so. 

• * Simulation limitation, no present pos report and BFT. 

• Simulation failure of high action display caused lower SA until pilot was able to ID 
problem and find correct info in other sources (TSD). 

• Overall: SA was much higher with UAS than without in this msn. Typically msn changes 
are the hardest task that a crew/team/company must perform. With the UAS, team was able 
to use standoff and develop the situation quicker than without the UAS. 

• Detennining the BDA of enemy vehicles destroyed and its position/location. 
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C-4 SME Situation Awareness Comments 

• TADS Operations were shed to operate UAS. 

• More SA was maintained. The crew used UAS efficiently to engage targets while 
maintaining SA. 

• Did not know if they had level 4 control or not. 

• CPG fired second MSL with TADS FXD FWD - did you use UAS distance and HOG for 
MSL constraints used TGT as an ACQ source. 

• Crew closed to under 1KM during gun engagement. Crew discuss to pursue target. 

• Circled closer to -1-3KM from UAV LOS. 

• Crew awareness of UAV location 64 - TGT - UAS location. 

• Crew lost SA on UAV location during mission during missile engagements. Missiles 6, 7, 
and 8 were all outside the laser offset performance. 

• Crew fired 3 missiles outside the 60deg gun target line. 

• Good overall SA. 

• Good. 

• Front seat had problems knowing whether or not he had lased a weapon during missile 
engagements. 

• Using the UAS acq should have been used to maintain TGT position - lase and store. 

• Maintained excellent SA on the battle and other elements. 

• Crew was prompted to hurry in when AA force holding. 

• Crew prompted during AWS running foe by wingman to hurry up. 

• Could not properly detennine there position and the position of the UAS prior to missile 
engagements. 

• HF missile engagement with AATF further downrange along GTL. 

• Crew members had difficulty tracking with UAS payload and the crew had difficulty 
determining the location in regards to the UAS. 

• Loitered over the targets. 

• Crew over flew enemy targets. Did not maintain standoff. 

• Aircrew flew within 1km of targets. 
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Appendix D. UAS-Crewstation Interface Ratings and Comments 


PI. How quickly were you able to navigate through the UAS PVI menu pages (below) to 
accomplish a task? 



Very 

Quickly 

Somewhat 

Quickly 

Borderline 

Somewhat 

Slowly 

Very 

Slowly 

UAS Flight Mode 
Options (HOLD, DEST, 
GUIDE, PROG) on 
TCDL Page 

100% 





UTIL 

80% 

20% 




PLAN 

60% 

20% 

20% 



CODE 

80% 

20% 




SET 

60% 

40% 




TSD 

60% 

40% 




VIDEO 

80% 

20% 





P2. How often did you forget the steps required for navigating through the UAS PVI menu 
pages to accomplish a task? 



Frequently 

Often 

Occasionally 

Never 

UAS Flight Mode 
Options (HOLD, DEST, 
GUIDE, PROG) on 
TCDL Page 



60% 

40% 

UTIL 



40% 

60% 

PLAN 


20% 

40% 

40% 

CODE 




100% 

SET 



40% 

60% 

TSD 



20% 

80% 

VIDEO 




100% 
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P3. How quickly were you able to use the switches on the grips and displays to control the 
UAS? 



Very 

Quickly 

Somewhat 

Quickly 

Borderline 

Somewhat 

Slowly 

Very 

Slowly 

UAS FOV 

40% 

60% 




Map Symbols 

75% 

25% 




FLIR Polarity 

100% 





Sensor Slave 

60% 

40% 




Manual Tracker 

40% 

40% 


20% 


Laser Trigger 

80% 

20% 




Sight Select 

80% 

20% 





If you answered “Somewhat Slowly”, “Very Slowly”, “Frequently”, or “Often” to any of the 
questions, explain why you had problems (e.g., “navigating through the UAS menu pages was a 
slow process due to having to page through too many display screens”). 

• Plan- was similar to RTE menu page and thought selecting “plan 2” was enough. Kept 
forgetting to press R4 on top page to select plan. 

• Manual tracker on system; rate should be similar to TADS. 

• Lack of exercise and training. 

P4. Would you be able to more quickly and easily control the UAS if the sensor image was 
displayed on the TED AC vs. the right MPD? 

Yes - 20% No- 80% 

If yes, explain why it would be quicker/easier to control the UAS on the TED AC: 

• Physiologically it would be easier. Head is not turned towards right MPD constantly. It 
would also be natural for the CPG to use the TED AC. 

P5. Did you experience any problems switching between the operation of the MTADS/PNVS 
and the UAS sensor? 

Yes - 60% No - 40% 

If yes, explain the problems: 

• Just had to develop muscle memory. 

• Just a momentary freeze and lock up. CPG needs to double check (usually) to ensure the 
correct sensor needed was displayed. 

• Lack of exercise (muscle memory). 
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P6. Was there any UAS symbology or wording depicted on the displays that was difficult to 
quickly and easily understand? 

Yes -40% No - 60% 

If yes, explain which symbology or wording was difficult to understand and why: 

• UAV field of regard is tough and unnecessary with TSD depiction. 

• The North (N) symbology should be a little brighter for SA. Occasionally, it was hard to 
break out from the video context. 

P7. On average, how often did the design of the UAS PCI significantly hinder you from quickly 
and easily controlling the UASs? (Circle one) 

12 3 4 

Never - 20% Seldom - 80% Often Frequently 

If ‘Often or Frequently’, describe how the design of the UAS PCI significantly hindered you 
from quickly and easily controlling the UASs: 

• Position of thumb force controller and UAS (FCR) FOV Switch. Again, with exercise 
(TFC Practice) this did not pose a significant problem. 

P8. On average, how often did the design of the UAS PVI contribute to high workload when 
controlling the UASs? (Circle one) 

12 3 4 

Never - 20% Seldom - 60% Often - 20% Frequently 

If ‘Often or Frequently’, describe how the design of the UAS PVI contributed to high workload 
during missions: 

• Almost too many holding options. 

• TFC sensitivity seemed too high in the RACR. 

P9. Did you feel that lack of linear motion compensation (LMC) for the UAS sensor increased 
the time required to detect, acquire and engage targets vs. having LMC for the UAS sensor? 

Significantly Increased Time - 60% 

Somewhat Increased Time - 40% 

No Difference - 0% 
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If you checked ‘Significantly or Somewhat Increased Time’, explain: 

• On long range 7000+ moving tgt was hard to track. Had to IAT terrain to have any fire 
control, which didn’t leave IAT for TGT. 

• It took longer to acquire multiple targets because it forced you to use IAT which limited 
your ability to pan to the next target. 

• LMC works great all the time for both TGT detection and engagement. I think it should 
have it. 

• Tracking and engaging the TGTS would be much easier with this capability. 

• With exercise/practice without LMC TFC is manageable. LMC will somewhat increase 
time due to stabilization/compensation of TFC. 

P10. List any other UAS PVI usability features that hindered your perfonnance during the 
missions: 

• Can’t fly UAV to TGT. 

• Can’t slave to WP/CM 

• Lack of TGT data number when utilizing TGT store procedures. Required CPG to 
navigate there other MPD pages to verify if TGT was stored and which are. 

0 CPG should be able to add points directly into the plan page of the UAS. 

0 FLT profiles of the UAS (Racetrack, Figure 8 and Orbit) should be changed to lower 
values or configurable in the aircraft UAS UTIL page. The size of these profiles can 
limit the use of the UAS laser when engaging TGTs with missiles. Aircraft (AH64) ma 
need to match the UAV’s position in order to meet missile constraints. (Too much 
moving around to meet this requirement). 

• Lack of visual cues on TSD which indicate constraints for Hellfire missile shots. (Ownship 
-> UAS tgt visual display). 

• Position of TFC and UAS FOV switch. 
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Appendix E. Top Improvements for UAS Integration 


Top Improvements Recommended By Pilots 

Improvements to UAS Cueing Functions/Symbology 

• When TADS LOS is slaved to UAV LOS and 30/60 safety/laser fans are within limits 
horizontal and vertical, LOS symbol should change to indicate within limits. SP or WP 
should know both alt, pos, range, and bearing and compute for crew. 

• Need symbology that allows 60deg “60-no” FQU for Hellfire 

• Slave UAV LOS to waypoint/control point 

• Allow UAV LOS to be slaved to any CM/WPT/HZ/TGT 

• More selectable UAS symbology. To aide with SA of UAS to our aircraft. 

• Allow all NAV points (WPT, HZ, CM) to be selectable for targeting the UAV sensor line 

of sight. AKA so I can have them for an acquisition source. 

• Have range info and heading of UAS on a TSD page, added to show page in attack. 

• Have range info and heading of UAS in HDU symbology. 

• UAS Laser line on TSD should be-(dashed) when not in range_(solid line) when 

in range and white when designating. 

• When you delete an IAT gate, the LOS should stay in the field of view and at the same spot 
it is when the gates are deselected. 

• UAS sensor indications on the TSD page (ex |-o<) like the FCR footprint. 

• A dashed or solid route while in a plan. 

• Add a show function on the TSD attack page to indicate heading and laser range from UAS 
to LOS. 

• Laser range to change color when active lasing, while leaving auto green. Possibly 
matching Arm color. 

• TGT data # symbology displayed on the UAS. 

• Laser designation window (basket) on TSD. MSL constraints box should be broken until 
UAS and aircraft are in the basket. Likewise the laser basket should be white until all 
constraints are met and then go green. 
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• When slaving UAS to a TGT, WP, HZ, or CM that info should appear next to the point on 
the UAS video. Likewise, shot at (X) should appear. 

• WPNS symbology (WAS’d) on display. 

• When range goes from auto to laser, range should change color. 

• Inverse symbology for UAV. 

• Shot file should say UAS if LOS is UAS, like it does for the FLIR. 

• UAS interface should work just like TADS and FCR. CAQing a TGT or icon on the TSD 
page should automatically make that TGT/icon the UAS AQC source. 

Improvements to Display Menu Pages 

• Ability to “Freeze” the UAS page (to include video) during Level 2 

• Divide symbology on UAS page to have all information that pertains to UAS up top of 
display, infonnation that pertains to own ship on bottom of display. 

• Allow declutter of UAS symbology. 

• When adding a waypoint, TGT, CM, be able to select GRID/Location by using the CAQ 
function. 

• Have UAV video have only LOS, Laser, Field of Regard, and HAD symbology. Just like 
the TADS for back seat. 

• When you store a target it should be hot key selectable as an acquisition source. 

• Add a fully functional COORD page to the TCDL page. 

• Be able to add WP/CM directly to plan page of the UAS using coordinate file or directly to 
plan page. 

• Options for missile codes. At least 2 selectable. 

• Add CORD access to UAS page at T-5 

• Allow for Show options on the UAS page to allow crew member to toggle on and off parts 
of the display. 

• LVL 3 / 4 -> Freeze/Replay function 

• Plan RTE building function enabling the cursor select option to input (WP, CM, HZ, TGT). 

Need Linear Motion Control (LMC) for UAS 

• LMC for UAV 
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LMC for UAS 


• Needs LMC for UAV. 

• LMC for UAS Sensor 

• Need LMC 

• UAS Sensor package needs more stabilization and green/solid box for in-constraints. 

Move UAS FOV switch to the Left Hand Grip 

• UAS FOV switch should be moved to the TADS FOV (Left Hand Grip) from the FCR 
FOV switch. This allows CPG to maintain manual track with right hand while changed 
FOV’s with left. 

• Re-positioning of UAS FOV switch to the LHG. 

Incorporate UAS Operator into Pre-Mission Planning 

• Incorporate UAS crewmember in mission planning and briefs. 

• Incorporate UAS ground operator with all inbriefs, FLT. Planning, etc., to ensure all 
available TM players are situationally aware as possible. That will lighten the load for the 
aircraft crewmembers. 

AMPS Integration 

• Ability to pre-plan multiple UAS systems in AMPS before launching our mission. 

• DTC/AMPS input for pre-mission planning. Routes, targets, etc... 

Other Improvements 

• Need the ability to send a UAS hellfire handover (UAS HO) to other aircraft. This UAS 
HO should constrain TGT, laser code UAS data. Once the A/C accepts the UAS HO, then 
the aircraft should automatically put the hellfire MSL in a remote mode. A display the laser 
basket (white/broken box for out of constraints and green/solid box for in-constraints.) 

• With sight select UAS, the TADS should go to fixed forward just like FCR and the li nk 
function should work just like FCR. 

• UAS hold pattern settings should allow for the lowest settings that each different UAS can 
perform. 

• Multiple IAT for UAS like MTADS. 

• Incorporate RFI with a UAS equipped aircraft. 

• Slew rates for UAS should match more close to TADS 

• allow UAV to fly to TGT. 
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Appendix F. Switch Actuations 


Table F-l. Switch actuations per mission. 



Number of Switch Actuations Per Mission 


Switch Description 

1 

2 

mm 

4 

5 

B 

7 

8 

9 

Bill 

Avg 

Std Dev 

Max 

Left Hand Grip TAD FOV Select 

El 

EM 

BEl 

El 

El 



EM 

268 


102.9 

93.72 

278 

Right Hand Grip Slave Select 

m 

MEM 

EH 

EH 

lEin 



EM 

232 


100.3 

84.79 

274 

Left Hand Grip Tracker Switch 

Eil 

IB 

El 

IEE1 

EM 



IEE1 

106 

158 

94.1 

53.17 

160 

Left Hand Grip Cursor Up/Down 

0 

0 


B 

EM 

B 

1 

107 

252 

201 

67.8 

92.13 

252 

CPG Left Bezel Buttons 


§m 

EH 

BEl 

KB 

BEl 

EM 

BEl 

39 

EH 

52.4 

14.78 

72 

Left Hand Grip Cursor Right/Left 

0 

0 

BEl 

B 

EH 

1 

i 

116 

181 

138 

50 

68.63 

181 

CPG Right Bezel Buttons 

EH 

EM 

msm 

KB 

EH 

EH 

EM 

EM 

29 

BEl 

44 

19.75 

78 

Right Hand Grip Laser Trigger 

EM 

EM 

EH 

EM 

El 

EH 

EH 

BEl 

47 

EH 

42.9 

10.28 

66 

Left Hand Grip Target Store 

IB 

EH 

El 

BEl 

0 

IB 

EM 

EH 

10 

EH 

19.8 

13.35 

46 

Left Hand Grip Weapon Select 

m 

IB 

EM 

MSM 

BEl 

B 

BEl 

8 

18 

4 

14.6 

8.21 

28 

UAV Field of View 

8 


0 

BEl 

EM 

EH 

B 

B 

6 

EH 

13.1 

12.58 

40 

TEDAC Gain Rocker Switch 

1 

0 

0 

0 

B 

BB 

B 

0 

0 

i 

1.7 

3.37 

11 

TEDAC TADS Video Select 

0 

0 

0 

0 

0 

0 

EH 

0 

2 

0 

1.6 

4.40 

14 

TEDAC Level Rocker Switch 

1 

0 

0 

0 

B 

B 

0 

0 

0 

B 

1.1 

1.91 

6 

TEDAC Display Brightness Rocker 

0 

0 

mm 

1 

i 

i 

i 

0 

0 

i 

0.7 

0.67 

2 

Left Hand Grip LMC Select 

mm 

0 

4 

0 

0 

0 

0 

0 

0 

0 

0.7 

1.49 

4 

TEDAC PNVS Video Select 

0 

0 

0 

0 

0 

0 

B 

0 

0 

0 

0.6 

1.90 

6 

TEDAC Display Contrast Rocker Switch 

1 

1 

0 

0 

0 

0 

0 

B 

0 

0 

0.5 

0.97 

3 

TEDAC Symbology Brightness Rocker 

0 

0 

0 

0 

0 

0 

0 

2 

0 

0 

0.2 

0.63 

2 

Left Hand Grip Scan Switch 

0 

0 

0 

0 

0 

0 

2 

0 

0 

0 

0.2 

0.63 

2 

TEDAC UAV Video Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Left Hand Grip Sensor Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Right Hand Grip Map Symbols Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Right Hand Grip Sight Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Right Hand Grip Enter for Cursor Control 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Right Hand Grip Polarity Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Sum 

345 

208 

429 

445 

599 

548 

872 

597 

1190 

BEE! 

609.2 

291.16 

1190 
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Table F-2. Time and percentage of time that menu pages were displayed on MPDs. 



CPG 

(Left MPD) 



CPG 

(Right 

MPD) 



PI 

(Left MPD) 



PI 

(Right MPD) 


Time (s) 

Title 

% 

Time (s) 

Title 

% 

Time (s) 

Title 

% 

Time (s) 

Title 

% 

28135.4 

TSD 

79% 

29678.1 

TCDL 

83% 

13306.6 

TSD 

37% 

18158.1 

TSD 

51% 

2173.8 

Wpn 

6% 

931.1 

Wpn 

3% 

8780.3 

TCDL 

25% 

12033.1 

TCDL 

34% 

1480.1 

Coord 

4% 

840.8 

TCDL Util 

2% 

3930.4 

Video 

11% 

1673.4 

Video 

5% 

1479.8 

Tre 

4% 

738.6 

Com 

2% 

3878.8 

Fit 

11% 

920.7 

Point 

3% 

848.9 

Point 

2% 

715.7 

TCDL Plan 

2% 

1967.7 

Eng 

6% 

839.9 

Eng 

2% 

467.1 

Eng 

1% 

709.1 

Fit 

2% 

956.1 

Wpn 

3% 

644.4 

Tre 

2% 

201.6 

Rpt 

1% 

598.3 

Menu 

2% 

584.1 

TCDL Plan 

2% 

497.8 

DTU 

1% 

156.2 

Map 

<1% 

509.7 

DTU 

1% 

520.7 

Com 

1% 

235 

Com 

1% 

143 

AC Util 

<1% 

493.2 

TSD 

1% 

467 

Tre 

1% 

202.2 

Coord 

1% 

132.6 

Shot 

<1% 

114.3 

Video 

<1% 

366.9 

Sight Boresight 

1% 

118.1 

TCDL Util 

<1% 

113.4 

Menu 

<1% 

93.8 

Tre 

<1% 

322.1 

Ctrlm 

1% 

82.7 

TCDL Plan 

<1% 

108.9 

Fit 

<1% 

63.3 

Freq 

<1% 

126.4 

WCA 

<1% 

82.5 

TSD Util 

<1% 

67.9 

Ctrlm 

<1% 

58.6 

Xpndr 

<1% 

109 

Coord 

<1% 

51.7 

Sight Boresight 

<1% 

49.4 

Show 

<1% 

42.5 

ASE 

<1% 

64.2 

Show 

<1% 

43.5 

Rpt 

<1% 

32.6 

Code 

<1% 

30.6 

Map 

<1% 

54.6 

Wpn Util 

<1% 

38.7 

Show 

<1% 

25.9 

TCDL Util 

<1% 

22.8 

Show 

<1% 

46.3 

AC Util 

<1% 

22.6 

Route Menu 

<1% 

23.5 

TCDL 

<1% 

21.5 

Route Menu 

<1% 

45.6 

Perf 

<1% 

19.9 

Wpn 

<1% 

19.3 

Route Menu 

<1% 

16.9 

Wpn Util 

<1% 

36.5 

TCDL Util 

<1% 

16.3 

Menu 

<1% 

17.2 

Wpthz 

<1% 

12.9 

Code 

<1% 

31.6 

Menu 

<1% 

15 

Man 

<1% 

13.2 

ADF 

<1% 

11.8 

Man 

<1% 

27.7 

Code 

<1% 

14 

ASE Util 

<1% 

12 

TSD Util 

<1% 

7.2 

AC Util 

<1% 

24 

ASE 

<1% 

13.1 

Wpthz 

<1% 

10.5 

ASE 

<1% 

7 

Coord 

<1% 

22.6 

TSD Util 

<1% 

9.1 

Net 

<1% 

9.6 

Video 

<1% 

6.9 

WCA 

<1% 

20.8 

Route Menu 

<1% 

6.4 

Edit 

<1% 

5.4 

Zn 

<1% 

5.2 

Fuel 

<1% 

18.4 

Fuel 

<1% 

6.1 

ASE 

<1% 

4.2 

Fault 

<1% 

5.2 

Perf 

<1% 

14 

Sys 

<1% 

2.8 

Map 

<1% 

4 

Wpn Util 

<1% 

4.1 

Sine 

<1% 

9.1 

Map 

<1% 

2.7 

Zn 

<1% 

3.6 

Chan 

<1% 

4 

Eng 

<1% 

8.2 

Edit 

<1% 

2 

DMS 

<1% 

3.5 

WCA 

<1% 

3.8 

TSD Util 

<1% 

7.7 

Point 

<1% 

1.1 

Fit 

<1% 

3.5 

Fuel 

<1% 

3.2 

HQ2 

<1% 

5.4 

Net 

<1% 




2.9 

DMS Util 

<1% 

1.2 

TCDL Code 

<1% 







2.3 

DMS 

<1% 

0.9 

DMS 

<1% 
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Table F-3. Time of movement of switches per mission. 




Time of Movement 

Column 

Detailed Description 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Avg 

Std Dev 

Max 

WpnTrig 

Weapon Trigger 

9.3 

14.1 

23.4 

8.8 

10.2 

3.4 

9.9 

2.4 

2.1 

7 

9.06 

6.3 

23.4 

TFCAz 

Right Hand Grip Thumb 

Force Controller Azimuth 

1387.7 

545 

901.7 

1232.1 

1255.3 

813.2 

966.5 

732.9 

959.7 

982.6 

977.67 

256.1 

1387.7 

TFCEI 

Right Hand Grip Thumb 

Force Controller Elevation 

1490.7 

531.5 

957.4 

1344.7 

1394 

832.9 

1022.3 

747.7 

1068.7 

990.8 

1038.07 

301.5 

1490.7 
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Intentionally Left Blank. 
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Acronyms 


AAR 

After Action Review 

AB3 

AH-64D Block III 

ACP 

Air Control Point 

AMPS 

Aviation Mission Planning System 

AOI 

area of interest 

ARH 

Armed Reconnaissance Helicopter 

ARL 

U.S. Army Research Faboratory 

ASL 

Applied Science Faboratories 

ATM 

Aircrew Training Manual 

BP 

Battle Position 

BWRS 

Bedford Workload Rating Scale 

CDD 

Capability Development Document 

CPC 

Comanche Portable Cockpit 

CPG 

Copilot-Gunner 

D 

Demand 

EA 

Engagement Area 

EDS 

Engineering Development Simulator 

EUD 

Early User Demo 

FAC 

Flight Activity Category 

FCR 

fire control radar 

FORSCOM 

Forces Command 

FOV 

Field of View 

HDU 

Helmet Display Unit 

HRED 

Human Research & Engineering Directorate 
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LEUE 

Limited Early User Evaluation 

LMC 

Linear Motion Control 

LOS 

line of sight 

MANPRINT 

Manpower and Personnel Integration 

MPD 

Multi-Purpose Display 

MTADS 

Modernized Target Acquisition Detection System 

NAI 

Named Area of Interest 

NOE 

Nap-of-Earth 

OneSAF 

One Semi-Automated Forces 

OTW 

Out-the-Window 

PI 

Pilot (pilot in back seat who flew the aircraft) 

PMO 

Product Manager’s Office 

PNVS 

pilot night vision sensor 

RACRS 

Risk and Cost Reduction Simulator 

RL 

Readiness Level 

RP 

Release Point 

S 

Supply 

SASA 

Situation Awareness 

SART 

Situational Awareness Rating Technique 

SME 

Subject Matter Expert 

SSQ 

Simulator Sickness Questionnaire 

TADS 

Target Acquisition and Designation System 

TCDL 

Tactical Common Data Link 

TCMRA 

TRADOC Capability Manager, Reconnaissance Attack 

TCM 

TRADOC Capability Manager 

TEDAC 

TADS Electronic Display and Control 

TFC 

Thumb Force Control 
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TRADOC 

Training and Doctrine Command 

TS 

Total Severity 

UP 

tactics, techniques, and procedures 

U 

Understanding 

UAS 

Unmanned Aerial System 

UCI 

UAS-Crewstation Interface 

VFR 

Visual Flight Rules 

VMC 

Visual Meteorological Conditions 

WSRT 

Wilcoxon Signed Ranks Test 
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NO. OF 

COPIES ORGANIZATION 


NO. OF 

COPIES ORGANIZATION 


1 DEFENSE TECHNICAL 
(PDF INFORMATION CTR 
only) DTIC OCA 

8725 JOHN J KINGMAN RD 
STE 0944 

FORT BELVOIR VA 22060-6218 

1 DIRECTOR 

US ARMY RESEARCH LAB 
IMNE ALC IMS 
2800 POWDER MILL RD 
ADELPHI MD 20783-1197 

1 DIRECTOR 

US ARMY RESEARCH LAB 
AMSRD ARL Cl OK TL 
2800 POWDER MILL RD 
ADELPHI MD 20783-1197 

1 DIRECTOR 

US ARMY RESEARCH LAB 
AMSRD ARL Cl OK PE 
2800 POWDER MILL RD 
ADELPHI MD 20783-1197 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR ML J MARTIN 
MYER CENTER RM 2D311 
FT MONMOUTH NJ 07703-5601 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MZ A DAVISON 
320 MANSCEN LOOP STE 115 
FT LEONARD WOOD MO 65473 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MD T DAVIS 
BLDG 5400 RM C242 
REDSTONE ARSENAL AL 35898-7290 

1 COMMANDANT USAADASCH 
ATSA CD 

AMSRD ARL HR ME DR HAWLEY 
5800 CARTER RD 
FT BLISS TX 79916-3802 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MM DR V J RICE 
BLDG 4011 RM 217 
1750 GREELEY RD 
FT SAM HOUSTON TX 78234-5002 


1 ARMY RSCH LABORATORY - HRED 
ARMC FIELD ELEMENTT 
AMSRD ARL HR MH C BURNS 
THIRD AVE BLDG 1467B RM 336 
FT KNOX KY 40121 

10 HRED 

USAACE FIELD ELEMENT 
AMSRD ARL HR MJ D DURBIN 
BLDG 4506 (DCD) RM 107 
FT RUCKER AL 36362-5000 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MK MR J REINHART 
10125 KINGMAN RD 
FT BELVOIR VA 22060-5828 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MY M BARNES 
2520 HEALY AVE 
STE 1172 BLDG 51005 
FT HUACHUCA AZ 85613-7069 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MP D UNGVARSKY 
POPE HALL BLDG 4709 
BCBL 806 HARRISON DR 
FT LEAVENWORTH KS 66027-2302 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MJF J HANSBERGER 
JFCOM JOINT EXPERIMENTATION J9 
JOINT FUTURES LAB 
115 LAKEVIEW PKWY STE B 
SUFFOLK VA 23435 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MQ M R FLETCHER 
NATICK SOLDIER CTR 
AMSRD NSC WSE BLDG 3 RM 343 
NATICK MA 01760-5020 

1 ARMY RSCH LABORATORY-HRED 
AMSRD ARL HR MT J CHEN 
12423 RESEARCH PKWY 
ORLANDO FL 32826 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MT C KORTENHAUS 
12350 RESEARCH PKWY 
ORLANDO FL 32826 


1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MG R SPINE 
BLDG 333 

PICATINNY ARSENAL NJ 07806-5000 
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NO. OF 

COPIES ORGANIZATION 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MS C MANASCO 
SIGNAL TOWERS 
BLDG 29808A RM 303 
FT GORDON GA 30905-5233 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MU 
6501 E 11 MILE RD MS 284 
BLDG 200A 2ND FL RM 2104 
WARREN MI 48397-5000 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MF C HERNANDEZ 
2421 NW AUSTIN RD STE 220 
FT SILL OK 73503-9042 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MV 
S MIDDLEBROOK 
91012 STATION AVE RM 348 
FT HOOD TX 76544-5073 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MN R SPENCER 
DCSFDI HF 

HQ USASOC BLDG E2929 
FT BRAGG NC 28310-5000 

1 ARMY RSCH LABORATORY - HRED 
AMSRD ARL HR MW E REDDEN 
BLDG 4 ROOM 332 
FT BENNING GA 31905-5400 

1 ARMY G1 
(CD DAPE MR B KNAPP 
only) 300 ARMY PENTAGON RM 2C489 
WASHINGTON DC 20310-0300 

1 US ARMY RSRCH DEV & ENGR 
CMND 

SYS OF SYS INTEGRATION 
AMSRD SS T 
6000 6TH ST STE 100 
FT BELVOIR VA 22060-5608 


NO. OF 

COPIES ORGANIZATION 

ABERDEEN PROVING GROUND 
1 DIR USARL 

AMSRD ARL Cl OK TP (BLDG 4600) 

5 DIR USARL 

AMSRD ARL Cl OK TP 
S FOPPIANO 
AMSRD ARL HR 
T LETOWSKI 
AMSRD ARL HR MD 
D HARRAH 
AMSRD ARL HR MR 
F PARAGALLO 
AMSRD ARL HR SD 
B AMREIN 

TOTAL: 42 (1 PDF, 1 CD, 40 HCs) 
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